Streaming over UDP
The IQ-9075 enables developers to build video streaming solutions with hardware-accelerated encoding. This page provides verified example pipelines to send and receive H.264 encoded video through UDP. The performance results show that Dragonwing is capable of handling 4K@30fps UDP streams.
Problems running the pipelines shown on this page? Please see our
GStreamer Debugging guide for help
.
UDP Streaming Pipelines
This test validates basic video streaming over UDP using GStreamer. Instead of a real camera source, a synthetic video stream is generated using videotestsrc with a black pattern.
The setup consists of:
- Sender: Dragonwing 9075 EVK (Ubuntu, accessed via SSH)
- Receiver: Host PC on the same network
Make sure you have connection between board and host PC.
H.264 Streaming
Sender (Dragonwing EVK)
Replace with your host IP
HOST_IP=X.X.X.X
And choose one amongst the following:
Stream videotestsrc:
gst-launch-1.0 -e -v videotestsrc pattern=black is-live=true ! video/x-raw,format=NV12,width=1280,height=720,framerate=30/1 ! v4l2h264enc ! h264parse ! rtph264pay config-interval=1 pt=96 mtu=1400 ! udpsink host=$HOST_IP port=5000 sync=false async=false
Stream from camera connected to JCAM0:
gst-launch-1.0 -e qtiqmmfsrc camera=0 ! video/x-raw,format=NV12,width=1280,height=720,framerate=30/1 ! queue max-size-buffers=4 leaky=downstream ! v4l2h264enc qos=true ! video/x-h264,profile=high,stream-format=byte-stream,alignment=au ! h264parse ! rtph264pay config-interval=1 pt=96 mtu=1400 ! udpsink host=$HOST_IP port=5000 sync=true async=false
Stream from file:
gst-launch-1.0 filesrc location=camera-test2.mp4 ! qtdemux ! h264parse ! v4l2h264dec capture-io-mode=dmabuf output-io-mode=dmabuf ! video/x-raw,format=NV12,width=1280,height=720,framerate=30/1,interlace-mode=progressive,colorimetry=bt601 ! v4l2h264enc ! h264parse ! rtph264pay config-interval=1 pt=96 mtu=1400 ! udpsink host=$HOST_IP port=5000
Receiver (Host PC)
Please, choose one amongst the following:
Display transmission:
gst-launch-1.0 -v udpsrc port=5000 caps="application/x-rtp,media=video,encoding-name=H264,payload=96,clock-rate=90000" ! rtph264depay ! avdec_h264 ! videoconvert ! autovideosink sync=false
Receiver camera stream with jitter buffer:
gst-launch-1.0 -v udpsrc port=5000 caps="application/x-rtp,media=video,encoding-name=H264,payload=96" ! rtpjitterbuffer latency=50 ! rtph264depay ! avdec_h264 ! autovideosink sync=true
Save stream in file:
gst-launch-1.0 -e -v udpsrc port=5000 caps="application/x-rtp,media=video,encoding-name=H264,payload=96" ! rtpjitterbuffer latency=50 ! rtph264depay ! h264parse ! mp4mux ! filesink location=received.mp4
H.265 Streaming
Sender (Dragonwing EVK)
Replace with your host IP
HOST_IP=X.X.X.X
And choose one amongst the following:
Stream videotestsrc:
gst-launch-1.0 -e -v videotestsrc pattern=black is-live=true ! video/x-raw,format=NV12,width=1280,height=720,framerate=30/1 ! v4l2h265enc ! h265parse ! rtph265pay config-interval=1 pt=96 mtu=1400 ! udpsink host=$HOST_IP port=5000 sync=false async=false
Stream from camera connected to JCAM0:
gst-launch-1.0 -e qtiqmmfsrc camera=0 ! video/x-raw,format=NV12,width=1280,height=720,framerate=30/1 ! queue max-size-buffers=4 leaky=downstream ! v4l2h265enc qos=true ! video/x-h265,profile=main,stream-format=byte-stream,alignment=au ! h265parse ! rtph265pay config-interval=1 pt=96 mtu=1400 ! udpsink host=$HOST_IP port=5000 sync=true async=false
Stream from file:
gst-launch-1.0 filesrc location=input_h265.mp4 ! qtdemux ! h265parse ! rtph265pay config-interval=1 pt=96 mtu=1400 ! udpsink host=$HOST_IP port=5000
Receiver (Host PC)
Please, choose one amongst the following:
Display transmission:
gst-launch-1.0 -v udpsrc port=5000 caps="application/x-rtp,media=video,encoding-name=H265,payload=96,clock-rate=90000" ! rtph265depay ! avdec_h265 ! videoconvert ! autovideosink sync=false
Receiver camera stream with jitter buffer:
gst-launch-1.0 -v udpsrc port=5000 caps="application/x-rtp,media=video,encoding-name=H265,payload=96" ! rtpjitterbuffer latency=50 ! rtph265depay ! avdec_h265 ! autovideosink sync=true
Save stream in file:
gst-launch-1.0 -e -v udpsrc port=5000 caps="application/x-rtp,media=video,encoding-name=H265,payload=96" ! rtpjitterbuffer latency=50 ! rtph265depay ! h265parse ! mp4mux ! filesink location=received.mp4
Performance testing
This table summarizes performance measurements of a GStreamer H.264 encoding pipeline using the perf element. The goal of these tests is to evaluate the impact of input video format and resolution on throughput (FPS) and CPU usage.
All tests were executed using a live videotestsrc at 30 FPS, encoded with v4l2h264enc and streamed over RTP. The perf element was used to monitor pipeline framerate and overall CPU load.
With videotestsrc
Pipeline used:
gst-launch-1.0 -e videotestsrc is-live=true ! video/x-raw,format=NV12,width=$WIDTH,height=$HEIGHT,framerate=30/1 ! queue max-size-buffers=4 leaky=downstream ! perf print-cpu-load=1 ! v4l2h264enc qos=true ! video/x-h264,profile=high,stream-format=byte-stream,alignment=au ! h264parse ! rtph264pay config-interval=1 pt=96 mtu=1400 ! fakesink
| Test # | Format | Resolution | %MEM | RSS (KB) | BPS | Mean BPS | FPS | Mean FPS | CPU (%) |
|---|---|---|---|---|---|---|---|---|---|
| 1 | NV12 | 640x480 | 0.0 | 16320 | 114786304.000 | 112317781.333 | 29.997 | 29.985 | 2 |
| 2 | NV12 | 1280x720 | 0.0 | 19012 | 339148800.000 | 339148800.000 | 29.987 | 29.988 | 3 |
| 3 | NV12 | 1920x1080 | 0.0 | 25100 | 752025600.000 | 752989735.385 | 29.990 | 29.997 | 4 |
| 4 | NV21 | 640x480 | 0.0 | 16988 | 111083520.000 | 111083520.000 | 30.000 | 29.980 | 2 |
| 5 | NV21 | 1280x720 | 0.0 | 19008 | 339148800.000 | 339148800.000 | 29.972 | 29.988 | 2 |
| 6 | NV21 | 1920x1080 | 0.0 | 25208 | 752025600.000 | 752025600.000 | 30.001 | 29.980 | 4 |
With camera
Pipeline tested:
gst-launch-1.0 -e qtiqmmfsrc camera=0 ! video/x-raw,format=NV12,width=1280,height=720,framerate=30/1 ! queue max-size-buffers=4 leaky=downstream ! perf print-cpu-load=1 ! v4l2h264enc qos=true ! video/x-h264,profile=high,stream-format=byte-stream,alignment=au ! h264parse ! rtph264pay config-interval=1 pt=96 mtu=1400 ! fakesink
| Resolution | %MEM | RSS (KB) | BPS | Mean BPS | FPS | Mean FPS | CPU (%) |
|---|---|---|---|---|---|---|---|
| 1280x720 | 0.2 | 76192 | 339148800.000 | 339384320.000 | 29.951 | 29.978 | 13 |
| 1920x1080 | 0.2 | 75860 | 339148800.000 | 339148800.000 | 30.120 | 29.958 | 13 |
| 3840x2160 | 0.2 | 75852 | 339148800.000 | 335918811.429 | 29.944 | 29.663 | 12 |
Element latency
| element | count | avg_ms | min_ms | max_ms | p50_ms | p90_ms | p95_ms | p99_ms |
|---|---|---|---|---|---|---|---|---|
| v4l2h264enc0 | 68 | 3.088 | 2.460 | 8.308 | 3.016 | 3.252 | 3.333 | 3.859 |
| queue0 | 68 | 0.398 | 0.080 | 0.783 | 0.444 | 0.641 | 0.685 | 0.706 |
| h264parse0 | 68 | 0.158 | 0.108 | 1.002 | 0.127 | 0.192 | 0.199 | 0.283 |
| rtph264pay0 | 68 | 0.102 | 0.045 | 0.209 | 0.086 | 0.149 | 0.176 | 0.209 |
| perf0 | 68 | 0.097 | 0.038 | 0.454 | 0.082 | 0.110 | 0.119 | 0.420 |
| capsfilter0 | 68 | 0.081 | 0.052 | 0.119 | 0.081 | 0.090 | 0.109 | 0.114 |
| capsfilter1 | 68 | 0.072 | 0.047 | 0.130 | 0.065 | 0.093 | 0.097 | 0.109 |
The total average latency of the pipeline from the moment a buffer leaves the camera source pad until it leaves the payloader source pad is nearly 4 ms. Note that this total latency is not the glass to glass latency.
| metric | value |
|---|---|
| TOTAL_AVG_SUM_MS | 3.996 |
| TOTAL_AVG_SUM_NS | 3995529 |
FAQ
- Is it possible to capture and stream 4K@30fps through UDP on the IQ-9075?
- Yes, the performance results show that the CPU load is 12% for this case and that the pipeline can maintain 30fps.
- Is RTP over UDP encrypted?
- Not by default, but this can be achieved.
Related pages
- Qualcomm Dragonwing Technical Guide/Multimedia Support/Streaming
- Qualcomm Dragonwing Technical Guide/Multimedia Support/Streaming/RTSP
- Qualcomm Dragonwing Technical Guide/Multimedia Support/Video/Encoding
- Qualcomm Dragonwing Technical Guide/Multimedia Support/Video/Decoding